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Access  to  sustainable  and  affordable  energy  services  is  a  crucial  factor  in  reducing  poverty  in  developing 
countries.  In  particular,  small-scale  and  community-based  renewable  energy  projects  are  recognized  as 
important  forms  of  development  assistance  for  reaching  the  energy  poor.  However,  to  date  only  a  few 
empirical  evaluations  exist  which  analyze  and  compare  the  impact  of  these  projects  on  local  living 
conditions  and  their  sustainability  ex-post  implementation. 

To  better  understand  the  impacts  and  the  conditions  that  influence  sustainability  of  these  projects, 
the  research  presented  in  this  paper  evaluated  23  local  development  projects  post  implementation. 
By  applying  an  standardized  evaluation  design  to  a  cross-sectional  sample  in  terms  of  renewable  energy 
sources  (solar,  wind,  biomass,  hydro),  user  needs  (electricity,  food  preparation,  lighting,  productive  uses), 
community  management  models,  finance  mechanisms  and  geographical  locations,  the  review  results 
provide  valuable  insights  on  the  underlying  conditions  that  influence  the  success  or  failure  of  these 
small-scale  local  energy  interventions.  The  empirical  evidence  suggests  that  the  sustainability  of  small- 
scale  energy  implementations  (  <  100  kW)  in  developing  countries  is  determined  by  the  same  factors, 
independent  of  the  socio-cultural,  political  and  ecological  context.  These  findings  allow  to  better  predict 
the  long-term  success  of  small  sustainable  energy  projects  in  developing  countries,  this  can  help  to 
improve  project  designs  and  increase  the  certainty  for  future  investment  decisions. 
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1.  Introduction 

Over  the  last  two  decades  the  number  of  people  without  access 
to  modern  energy  services,  defining  those  lacking  access  to 
electricity  and  clean  cooking  facilities,  has  decreased  significantly. 
However,  2.6  billion  people  still  lack  access  to  affordable  and 
reliable  energy  services  to  meet  their  basic  energy  needs  [1],  Most 
of  these  people  live  in  rural  areas  in  developing  countries  or 
belong  to  the  urban  poor.  Without  access  to  energy  the  chances  of 
reducing  poverty  and  advancing  development  are  poor.  Thus,  even 
though  none  of  the  Millennium  Development  Goals  (MDGs), 
representing  the  universal  development  objectives  agreed  on  by 
the  international  community,  relate  directly  to  energy,  it  is  widely 
acknowledged  that  it  will  not  be  possible  to  achieve  the  MDGs 
without  expanding  energy  access  [2],  But  energy  access  is  not  the 
only  concern;  energy  supply  should  also  be  sustainable,  avoiding 
the  drawbacks  of  conventional  energy  sources  such  as  harmful 
emissions,  noise,  high  fuel  costs  and  supply  insecurities.  This  call 
for  sustainable  energy  access  for  development  is  further  under¬ 
lined  by  the  declaration  of  the  decade  2014-2024  as  the  Decade  of 
Sustainable  Energy  for  All  by  the  United  Nations  General  Assembly. 
The  initiative  supports  renewable  energy  sources  as  a  key  tech¬ 
nology  to  reach  the  energy  poor,  offering  clean  electricity,  heating, 
cooking  and  lighting  solutions  to  people  and  communities  who 
currently  depend  on  traditional  energy  sources  and/or  expensive 
fossil  fuels.  Renewable  energy  technologies  are  regarded  as  parti¬ 
cularly  suitable  because  they  can  provide  small-scale  solutions  and 
decentralized  energy  supply  that  meet  the  needs  of  the  population 
most  widely  affected  by  energy  poverty.  Furthermore,  innovations 
and  cost  reductions  over  the  last  decade  have  made  renewable 
energies  more  economically  competitive  in  relation  to  traditional 
fuels  [3],  which  have  also  helped  to  strengthen  the  case  for 
renewables.  Nevertheless,  these  technologies  still  face  a  range  of 
social,  economic  and  structural  challenges,  requiring  not  only 
further  technological  development  but  also  a  deeper  understand¬ 
ing  of  both  the  success  factors  and  the  barriers  to  accomplish 
widespread  dissemination.  This  is  demonstrated  by  a  recent  study 
by  Bhattacharyya  [4]  on  financing  energy  access  and  off-grid 
electrification,  which  showed  that  despite  recent  progress  and 
the  support  of  the  international  community  only  limited  funding  is 
available  for  small-scale  community  projects.  According  to  Bhat¬ 
tacharyya  [4]  most  funds  are  allocated  to  larger  energy  generation 
projects  that  barely  address  the  energy  needs  of  the  poor.  Yet, 
small-scale  projects  can  play  a  key  role  in  supporting  the  transi¬ 
tion  towards  more  sustainable  energy  systems.  Therefore,  the 
WISIONS  initiative1  provides  support  to  innovative  project 
approaches  and  capacity  development  to  respond  to  energy  needs 
at  local  level  via  its  Sustainable  Energy  Project  Support  (SEPS) 
scheme.  Since  2004,  a  total  of  64  projects  worldwide  have  been 
selected  for  SEPS  support.  These  projects  focus  on  different 
energy-related  needs,  technologies  and  implementation  concepts. 
Although  most  of  these  projects  were  completed  successfully, 
small-scale  renewable  energy  projects  do  not  automatically  become 
sustainable  in  the  long  term.  In  the  literature  there  are  numerous 
documented  accounts  of  development  efforts  that  fail  because  they 
cannot  create  conditions  that  lead  to  lasting  results  [5-7],  A  study 


1  "WISIONS  of  sustainability”  is  an  initiative  by  the  Wuppertal  Institute 
supported  by  the  Swiss-based  foundation  ProEvolution.  It  was  launched  in  2004 
to  promote  practical  and  sustainable  energy  projects.  To  ensure  the  sustainable 
character  of  the  projects  supported  by  the  SEPS  scheme  their  selection  is  based  on 
the  following  set  of  criteria:  technical  viability,  economic  feasibility,  local  and  global 
environmental  benefits,  replicability  and  marketability,  potential  for  poverty 
reduction,  social  equity  and  gender  issues,  local  involvement  and  employment 
potential,  sound  implementation  strategy  and  dissemination  concept.  For  more 
detailed  information  on  the  program,  please  visit  the  website  www.wisions.net. 


by  Bhattacharyya  [8]  on  the  relationship  between  energy  access 
programs  and  sustainable  development  suggests  that  the  existing 
practices  of  providing  energy  access  are  generally  unsustainable 
from  a  number  of  perspectives.  Therefore  it  is  important  to  evaluate 
and  accurately  assess  the  impact  and  sustainability  of  such  projects 
after  the  initial  project  activity  is  completed,  to  learn  from  results 
and  improve  the  quality  of  future  decisions  and  projects. 

In  view  of  these  observations,  the  main  objectives  of  this  paper 
are:  (a)  to  evaluate  the  impact;  and  (b)  to  determine  the  mid-term 
sustainability  of  23  small-scale  renewable  energy  projects  ex-post 
implementation  as  well  as  (c)  advancing  the  knowledge  of  the 
effects  of  energy  access  projects  at  local  level  by  exploring 
whether  influencing  factors  and  barriers  are  linked  to  the  type  of 
technology,  whether  they  depend  on  the  economic,  social  or 
geographical  background  or  whether  common  patterns  indepen¬ 
dent  of  these  factors  can  be  identified.  The  results  offer  stake¬ 
holders  information  about  the  major  influencing  factors  in  the 
success  or  failure  of  achieving  sustainability  in  small-scale  renew¬ 
able  energy  projects  in  developing  countries. 

The  article  is  organized  as  follows:  Section  2  presents  the 
background  for  evaluating  small-scale  renewable  energy  projects, 
Section  3  describes  the  methods  applied,  evaluation  design  and 
evaluation  sample,  Section  4  discusses  the  results  of  the  impacts 
and  sustainability  evaluation,  Section  5  summarizes  the  research 
outcomes  and  the  recommendations  these  results  imply  and  the 
paper  conclude  with  Section  6  that  discusses  the  limitations  and 
future  research  needs. 


2.  Overview:  the  state  of  evaluating  small-scale  renewable 
energy  projects  in  developing  countries 

Over  the  last  decade  the  international  community  has  empha¬ 
sized  the  importance  of  evaluating  development  interventions. 
This  is  reflected  in  the  increasing  attention  the  subject  receives  in 
publications  of  donor  organizations  and  research  institutions  on 
how  to  evaluate  projects  [9-15].  As  a  result,  many  evaluations  of 
individual  projects  or  country  programs  are  documented  in  the 
literature.  However,  only  very  few  studies  actually  evaluate  small- 
scale  community  projects  ( <  100  kW)  in  developing  countries 
with  regards  to  their  impact  on  local  living  conditions  and 
sustainability  post-installation  [16-19],  This  is  in  line  with  the 
report  from  the  UNDP  [20]  stating  that  studies  on  drivers  of 
success  and  sustainability  of  small-scale  projects  limited  to  a  small 
number  of  case  studies.  Ferrer-Marti  et  al.  [16]  who  provide  an 
overview  of  the  few  studies  that  have  addressed  these  subjects. 
Other  examples  include  Hong  and  Abe's  study  [17]  on  the 
sustainability  of  off-grid  rural  electrification  on  the  Pangan-an 
island  in  the  Philippines,  the  analysis  of  lessons  learned  from 
small-scale  bioenergy  projects  in  rural  China  by  Han  et  al.  [21  ],  the 
paper  by  Ilskog  and  Kjellstrdmb  [22]  assessing  rural  electrification 
cases  by  means  of  indicators  as  well  as  Balkema  et  al.  [23  who 
conduced  an  impact  assessment  of  several  small  scale  renewable 
energy  projects  in  developing  countries  based  on  the  MDGs,  could 
be  added  to  this  list. 

Nearly  all  of  these  studies  focus  either  on  one  technology  or  on 
a  specific  geographical  context.  This  corresponds  to  the  results  on 
rural  electrification  projects  in  general  by  Schillebeeckx  et  al.  [24] 
who  carried  out  a  content  analysis  of  232  papers  on.  This  analysis 
showed  a  clear  dominance  of  research  on  technology  and  institu¬ 
tional  contexts  while  less  attention  has  been  given  to  under¬ 
standing  user  needs.  With  regards  to  post-implementation  studies 
to  the  best  of  the  authors’  knowledge  there  have  been  almost  no 
studies  that  review  the  sustainability  of  small-scale  projects 
that  are  cross-sectional  in  terms  of  technologies,  user  needs 
and/or  geographical  regions.  An  exception  is  the  meta-analysis 
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from  Brass  et  al.  [19],  which  reviews  case  studies  of  small-scale  local 
energy  systems  from  existing  literature  with  regards  to  factors  that 
affected  the  outcomes  of  these  projects.  The  other  cross-sectional 
evaluations  that  exist  have  mostly  been  undertaken  by  donor 
organizations  such  as  the  World  Bank  [25],  FAO  [26],  UNDP  [20] 
or  the  Japan  International  Cooperation  agency  [27],  These  evalua¬ 
tions  have  focused  on  national  programs  or  projects  on  larger  scales 
and  with  higher  budget  s.  However,  these  studies  have  served  to 
demonstrate  that  the  evaluation  of  a  number  of  projects  imple¬ 
mented  within  a  common  framework,  but  under  diverse  contextual 
factors  can  provide  recommendations,  which  are  relevant  across 
project  boundaries.  Consequently,  the  authors  of  this  paper  are 
convinced  that  despite  the  methodological  challenges,  which  are 
discussed  in  Section  3.2,  a  cross-sectional  evaluation  can  create 
synergy  effects  and  provide  better  insights  into  factors  that  influ¬ 
ence  the  success  and  sustainability  of  small-scale  renewable  energy 
projects  in  developing  countries. 

The  Millennium  Development  Goals  were  chosen  as  reference 
frame  for  the  impact  assessment  as  they  represent  the  benchmark 
for  meeting  the  most  pressing  challenges  of  out  time  climate 
change  and  global  poverty.  Both  of  these  challenges  are  inter¬ 
related  and  linked  to  energy.  Not  to  include  energy  considerations 
in  development  projects  will  severely  limit  the  ability  to  achieve 
the  MDGs.  Therefore,  the  MDGs  are  widely  used  as  connectional 
framework  for  sustainable  energy  development  interventions. 


3.  Methods 

3.1.  Evaluation  principles 

In  the  growing  field  of  evaluation  practices  in  development 
activities  it  is  important  to  determine  the  terms  and  objectives  of 
an  evaluation.  A  set  of  five  key  evaluation  criteria  to  examine 
development  assistance  was  established  by  the  Development 
Assistance  Committee  (DAC)  and  is  widely  applied  by  donor 
countries  and  multilateral  organizations.  These  five  core  criteria 
are  [28]:  relevance,  effectiveness,  efficiency,  impact  and  sustain¬ 
ability.  While  relevance,  effectiveness  and  efficiency  should  be 
analyzed  in  the  earlier  steps  of  the  project  cycle,  impact  and 
sustainability  can  and  should  be  assessed  post  implementation. 
These  type  of  ex-post  evaluations  are  particularly  helpful  in 
answering  the  question  of  what  works  and  why  and,  consequently, 
can  guide  future  improvements  in  project  design  and  practice  by 
identifying  success  factors  and  explaining  failure  [29],  Accordingly 
this  paper  focuses  on  reviewing  (a)  impact  and  (b)  sustainability. 

(a)  Impact  is  understood  to  mean  both  positive  and  negative 
change  produced  directly  or  indirectly,  intentionally  or  unin¬ 
tentionally,  by  a  development  intervention.  This  includes  the 
main  impacts  and  effects  resulting  locally  from  the  activity  in 
terms  of  social,  economic,  environmental  and  other  develop¬ 
ment  indicators  [28],  The  impact  evaluation  should  deliver  a 
balanced  assessment  of  whether  the  program  produced  the 
desired  results  for  individuals,  households  and  institutions  and 
whether  those  effects  can  be  attributed  to  the  program 
intervention  [9[.  These  assessments  should  consider  contex¬ 
tual  factors  such  as  social,  cultural,  economic  and  political 
aspects  as  critical  conditions  that  shape  the  nature  of  devel¬ 
opment  [5], 

To  assess  the  impact  the  international  working  group  for 
Monitoring  and  Evaluation  in  Energy  for  Development  (M&EED) 
suggests  that  the  MDGs  should  be  used  as  reference  as  they 
represent  the  international  priorities  for  sustainable  develop¬ 
ment  [30],  Consequently,  the  findings  of  this  evaluation  are 
assessed  on  the  basis  of  their  contribution  to  attaining  the  MDGs. 


Although,  in  a  general  sense,  extending  energy  services  can 
contribute  towards  all  the  development  goals,  it  was  not  possible 
to  measure  all  of  these  impacts  through  the  applied  evaluation 
design.  To  account  for  all  impacts,  detailed  on-site  studies  would 
have  been  necessary  for  each  of  the  23  projects.  Due  to  time  and 
financial  restrictions  this  was  not  possible  for  all  projects.2 
Therefore,  the  impact  assessment  was  concentrated  on  impacts 
related  to  the  MDGs  presented  in  Table  1,  which  could  be 
measured  with  the  applied  approach.  Table  1  also  gives  an 
overview  of  the  impact  categories  that  reflect  the  contribution 
made  by  the  projects  to  achieving  the  selected  MDGs. 

(b)  Sustainability  in  the  present  case  is  concerned  with  measuring 
whether  the  expected  benefits  of  a  project,  i.e.  access  to  clean 
energy  services,  business,  employment  and  training  opportu¬ 
nities,  food  security  etc.,  are  likely  to  persist  after  donor 
funding  has  been  withdrawn.  Sustainability  can  be  assessed 
by  answering  the  following  two  questions  (1)  to  what  extent 
did  the  benefits  of  a  project  continue  after  donor  funding 
ceased?  and  (2)  what  were  the  major  factors  which  influenced 
the  achievement  or  non-achievement  of  sustainability  of  the 
project?  [26],  The  timeframe  for  this  type  of  post  implementa¬ 
tion  evaluation  varies  but  in  most  cases  project  performance  is 
addressed  within  two  or  three  years  after  the  installation  [19], 
This  paper  evaluates  projects,  with  an  average  duration  of 
12-24  month,  that  were  started  between  2004  and  2008. 
Clearly  this  timeframe  is  insufficient  for  claiming  long-term 
success,  but  the  fact  that  the  technology  is  still  functioning  and 
being  used  2-8  years  after  its  initial  introduction  can  indicate 
whether  long-term  sustainability  is  likely  to  be  accomplished. 


3.2.  Evaluation  design 

The  study  is  based  on  empirical  data  from  the  conducted  survey 
and  secondary  data  from  literature,  national  statistics  and  project 
progress  and  final  reports.  The  evaluation  process  itself  was  designed 
as  semi-structured  in-depth  interviews  with  the  organizations  that 
implemented  and  monitored  the  initial  project  activities.  This  survey 
approach  was  chosen  as  it  is  time-effective  and  particularly  suitable 
for  addressing  questions  about  why  certain  decisions  are  made  and 
why  some  processes  work  better  than  others  [9[.  This  approach 
inevitably  has  limitations,  namely  the  problems  of  generalizing  the 
results  along  with  the  possibility  that  the  information  provided  is 
biased.  The  challenge  of  comparing  and  generalizing  the  outcomes 
was  helped  by  the  fact  that  all  projects  were  supported  under  the 
same  program  and  were,  therefore,  selected  based  on  the  same 
criteria.  This  means  that  a  number  of  variables  remain  constant, 
providing  a  sound  foundation  for  identifying  common  processes  and 
impacts  [12].  With  regards  to  the  possibility  of  predisposed  inter¬ 
pretations,  we  tried  to  limit  the  risk  by  validating  the  information 
through  comparing  the  reported  data  with  existing  project  docu¬ 
mentation  in  the  form  of  initial  project  design,  progress  and  final 
reports,  as  well  as  secondaiy  literature  and  statistics. 

Although  the  general  structure  of  the  questionnaire  was  identical 
and  key  issues  were  addressed  in  all  interviews,  not  all  questions 
were  relevant  to  the  entire  sample,  accordingly  the  questions  varied 
to  a  certain  extent  depending  on  the  project  design  and  technology. 
The  main  categories  addressed  were  (a)  overall  project  sustainability, 
(b)  technology,  (c)  social  and  economic  aspects,  (d)  environment, 
(e)  replication  and  dissemination  and  (f)  policy  development. 


2  While  for  larger  projects  the  resources  necessary  for  a  detailed  and  effective 
evaluation  amounted  to  only  0.2-1.25%  of  the  total  project  cost  [9,31],  this 
percentage  is  much  higher  for  small  projects  with  project  budgets  up  to  €100,000. 
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Table  1 

Selected  MDG  benefits  of  renewable  energy  projects. 


MDGs 


Impact  of  projects  on  MDGs 


MDG  1 :  Reducing  extreme  poverty  and  hunger 


MDG  7:  Ensuring  environmental  sustainability 


MDG  8:  Promoting  global  partnership  for  development 


•  Access  to  energy 

•  Productive  uses  of  energy 

•  Reduction  in  energy  expenditure 

•  Employment  and  training 

•  Improved  quality  of  life 

•  Reduced  GHG  emissions 

•  Unsustainable  energy  sources  replaced 

•  Policy  interactions 

•  Development  of  network  connections  and  partnerships 

•  Development  of  institutional  frameworks 

•  Dissemination 


Based  on  [20]  and  [24], 


Table  2 

Overview  projects  and  status. 


Technology 

Need/application 

Country 

Status  2012 

1 

Micro-hydro  power 

Electrification 

Brazil 

Fully  operational 

2 

Solar  PV 

Electrification  and  lighting 

Namibia 

Fully  operational 

3 

Pico-hydro  power 

Electrification  and  lighting 

Philippines 

Fully  operational 

4 

Improved  cooking  stoves 

Food  preparation 

Laos 

Fully  operational 

5 

Biogas 

Food  preparation 

India 

Fully  operational 

6 

Improved  cooking  stoves 

Food  preparation 

China 

Fully  operational 

7 

Biogas 

Food  preparation  and  heating 

Jordan 

Fully  operational 

8 

Efficient  pumps 

Irrigation 

India 

Fully  operational 

9 

Solar  PV  and  efficiency  improvement 

Lighting 

Mauritius 

Fully  operational 

10 

Solar  PV 

Lighting 

Kenya 

Fully  operational 

11 

Efficiency  improvement 

Lighting 

Mexico 

Fully  operational 

12 

Wind  power 

Electrification 

Peru 

Mostly  operational 

13 

Solar  cookers  and  improved  cooking  stoves 

Food  preparation 

Guatemala 

Mostly  operational 

14 

Biogas 

Food  preparation 

Latin  America 

Mostly  operational 

15 

Solar  cookers 

Food  preparation 

Nepal 

Mostly  operational 

16 

Solar  PV  and  wind  power 

Irrigation 

Tanzania 

Mostly  operational 

17 

Efficient  lanterns 

Lighting 

Sri  Lanka 

Mostly  operational 

18 

Solar  bakery 

Food  preparation 

Cameroon 

Mostly  operational  (2011) 

19 

Biogas 

Electrification 

Sri  Lanka 

Only  functioning  to  a  limited  extent 

20 

Solar  PV  and  micro-hydro  power 

Electrification 

Peru 

Only  functioning  to  a  limited  extent 

21 

Liquid  biofuel  Jatropha 

Irrigation 

Nepal 

Only  functioning  to  a  limited  extent 

22 

Liquid  biofuel  Jatropha 

Irrigation 

India 

Not  functioning 

23 

Biomass  gasification 

Industry 

India 

Not  functioning 

The  data  collected  consists  of  quantifiable  evidence,  such  as  the 
number  of  installations  or  number  of  beneficiaries,  as  well  as 
qualitative  aspects,  such  as  user  satisfaction,  impacts  on  society, 
awareness-raising  or  network  development. 

With  regards  to  the  impact  evaluation  it  was  not  possible  to 
establish  a  common  baseline  for  all  projects  to  measure  the  impacts 
due  to  the  small  number  of  projects  and  the  lack  of  detailed  socio¬ 
economic  background  data,  like  household  size,  income  levels, 
spending  habits,  education  levels,  energy  consumption  etc.,  for  the 
different  user  groups  in  the  individual  projects.  Instead  the  quasi- 
experimental  design  of  reflexive  comparison  was  applied,  comparing 
the  situations  of  the  participants  before  and  after  the  intervention  [9], 
To  minimize  the  risk  that  this  approach  implies,  the  issues  of  external 
effects  were  explicitly  addressed  in  the  interviews. 

3.3.  Evaluation  sample 

The  evaluation  cluster  consisted  of  23  projects,  supporting 
various  renewable  energy  technologies  including  solar,  wind,  hydro 
and  biomass  power,  as  well  as  efficiency  measures  to  meet  needs 
such  as  food  preparation,  lighting,  electrification  or  irrigation  in 


over  17  different  developing  countries  (Table  2).  About  a  quarter  of 
the  evaluated  projects  were  implemented  in  Sub-Saharan  Africa 
and  a  further  quarter  in  Latin  America,  while  the  other  half  were 
implemented  across  Asia,  with  one  additional  project  implemented 
in  the  Middle  East. 

The  most  common  application  of  the  implemented  technologies 
was  food  preparation,  followed  by  electrification  and  lighting. 
Irrigation  was  the  application  in  15%  of  the  reviewed  projects,  while 
“productive  use”  as  a  primary  application  played  only  a  minor  role. 
In  terms  of  technology,  the  applications  that  utilized  biomass  for 
energy  generation  in  the  broadest  sense  represented  the  largest 
group  (37%)  (Fig.  1 ).  The  second  largest  group  was  made  up  of  the 
technologies  that  transform  solar  radiation  into  energy,  such  as 
photovoltaic  or  solar  cookers  (30%).  Wind  and  hydro  power 
implementations  represented  only  7%  and  11%  respectively  of  the 
reviewed  projects.  Efficiency  measures  represented  15%  of  the 
implemented  technologies.  In  all  but  one  case  these  focused  on 
efficiency  improvements  for  lighting,  such  as  efficient  light  bulbs  or 
the  improved  use  of  daylight.  In  the  other  case  the  focus  was  on 
efficient  water  pumps  for  irrigation.  Improved  cooking  stoves  are 
classified  as  bioenergy  technology,  but  they  could  also  be  seen  as 
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efficiency  improvement  technologies;  this  classification  would  see 
the  share  of  efficiency  technologies  rise  to  26%. 

4.  Results 

4.1.  Impact 

When  examining  the  complex  circumstances  surrounding  local 
energy  access  and  energy  efficiency  projects  in  developing  coun¬ 
tries,  it  is  useful  to  consider  the  following  questions:  (a)  what  has 
happened  as  a  result  of  the  project  in  terms  of  measurable 
impacts?  And  (b)  what  real  difference  has  the  activity  made  to 
the  beneficiaries  in  terms  of  welfare  and  empowerment?  [28], 
In  the  following  section  these  questions  are  addressed  with  regards 
to  the  impact  the  23  renewable  energy  projects  have  had  on  the 
MDGs.  The  review  shows  that  most  projects  had  positive  impact 
on  factors  such  as  energy  access,  energy  costs,  employment, 
health,  communication  and/or  access  to  information,  with  the 
result  that  they  improved  the  quality  of  life  in  individual  house¬ 
holds  or  communities  and  contributed  to  the  achievement  of  the 
MDGs  (Table  3).  In  addition,  the  implementation  of  the  renewable 
energy  technologies  contributed  to  the  global  climate  change 
agenda,  although  these  effects  could  not  be  quantified  in  all  cases. 


25% 


Fig.  1.  Distribution  of  technologies  among  the  evaluation  sample  of  23  projects. 


Impact  on  economic  development  was  less  visible.  Most  projects 
were  focused  on  meeting  the  basic  energy  needs  of  the  poor,  such 
as  lighting,  cooking  etc.,  but  were  missing  out  on  providing  real 
business  development  opportunities.  The  review  further  revealed 
that  small-scale  energy  projects  can  have  an  impact  on  the 
development  of  energy  policies  if  the  implementing  organizations 
actively  promote  the  technologies  and  engage  with  the  govern¬ 
ment  organizations  responsible.  In  the  following  sections  these 
impacts  are  described  in  detail. 

4.3.3.  Impacts  of  projects  on  MDG  3;  reducing  extreme  poverty  and 
hunger 

As  previously  mentioned,  access  to  sustainable  and  affordable 
energy  services  is  a  crucial  factor  in  reducing  poverty  in  develop¬ 
ing  countries.  This  review  validates  that  small-scale  projects  can 
improve  and  increase  access  to  energy  for  individuals  and  com¬ 
munities  that  would  not  have  been  supplied  by  market  structures. 
This  is  similar  to  conclusions  drawn  by  other  evaluations  that 
focus  on  either  photovoltaic  [32]  or  wind  energy  [16],  Access  to 
energy  was  increased  through  the  project  activities  and  was  also 
further  extended  in  several  cases.  In  57%,  depending  on  the 
business  model,  additional  devices  were  either  installed,  whole¬ 
saled  or  additional  households  were  connected  after  the  initial 
project  was  completed.  Even  though  only  five  interview  partners 
were  able  quantify  the  increase,  these  five  projects  alone  provide 
350  additional  persons  with  clean  electricity,  heating,  cooking  or 
lighting  solutions.  And  up  to  2500  additional  people  will  further¬ 
more  gain  access  to  sustainable  energy  through  ongoing  up  scaling 
activities.  On  the  contrary  in  only  two  projects  the  number  of 
beneficiaries  decreased,  due  to  reasons  including  national  grid 
extension,  low  community  participation  or  unreliable  feedstock 
supply. 

However,  in  terms  of  productive  use  the  impacts  were  limited. 
Consequently,  the  evaluation  findings  support  the  results  of  other 
studies  on  rural  electrification  [25,20]  stating  that  although 
electrification  may  provide  opportunities  for  small  business  activ¬ 
ities,  productive  use  should  not  necessarily  be  anticipated  from 
small-scale  electrification  projects.  The  same  was  found  to  be  true 
for  the  cooking  and  lighting  projects.  This  means  that  even  though 
12  projects  mentioned  possible  productive  uses  of  energy,  the 
actual  number  of  people  that  used  the  energy  to  start  business 
activities  was  small.  Exceptions  were  the  irrigation  projects,  where 


Table  3 

Summary  of  impacts  on  selected  MDGs. 


MDG 


Impact  of  projects  on  MDG 


MDG  1 :  Reducing  extreme  poverty  and 
hunger 


•  Access  to  energy  was  increased  through  the  actual  projects  and  could  be  further  extended  in  nearly  50%  of  the  reviewed 
cases  through  the  installation/sale  of  additional  devices.  In  two  cases  the  number  of  beneficiaries  has  decreased 

since  the  implementation. 

•  12  projects  mentioned  productive  uses  of  the  provided  energy,  but  the  number  of  people  that  have  used  the  provided  energy 
services  to  start  business  activities  has  been  limited. 

•  Energy  expenditure  decreased  in  65%  of  the  23  projects. 

•  The  renewable  energy  projects  provided  additional  training  and  employment  opportunities  for  at  least  66  people. 

•  Most  projects  improved  living  conditions  by  reducing  health  risks,  providing  access  to  communication  and  information 
services,  providing  better  learning  environments,  reducing  expenditure  or/and  the  daily  workload. 


MDG  7:  Ensuring  environmental 
sustainability 


•  Quantification  was  not  possible,  but  the  evaluation  results  support  the  assumption  that  additional  amounts  of  GHG  emissions 
were  avoided  and  further  unsustainable  energy  sources  were  replaced. 


MDG  8:  Promoting  global  partnership 
for  development 


•  On  the  one  hand  national  grid  extension  programs  had  severe  negative  impacts  on  four  projects.  On  the  other  hand  at  least 
three  projects,  in  some  way  or  another,  stimulated  the  decision  to  introduce  support  schemes  for  renewable  energies 

at  national  level. 

•  In  44%  of  the  reviewed  projects  network  connections  between  stakeholders,  institutions  and  other  organizations 
were  developed  or  strengthened. 

•  Dissemination  strategies  were  generally  suitable  as  in  78%  of  the  cases  awareness  of,  and  interest  in,  renewable 
energy  technologies  was  raised. 

•  Replication  of  either  the  technology  or  management  model  was  achieved  in  40%  of  the  reviewed  cases. 
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higher  incomes  could  be  achieved  through  increased  agricultural 
productivity.  In  addition,  one  irrigation  project  triggered  business 
activity  in  the  form  of  the  establishment  of  a  tree  nursery 
producing  seedlings  that  are  sold  in  the  region  and  beyond  to 
farmer  groups,  companies  and  individuals. 

With  regards  to  socio-economic  benefits  the  review  revealed 
that  all  operational  projects  had  positive  impacts  for  the  local 
population.  The  most  mentioned  benefit  was  the  decrease  in 
expenditure  of  households  for  energy  carriers  like  fuel  wood, 
charcoal,  kerosene,  which  amounted  to  up  to  a  reduction  of  40% 
in  the  improved  cooking  stove  projects.  Overall,  energy  expendi¬ 
ture  decreased  in  65%  of  the  former  projects.  In  30%  of  these  cases 
the  change  in  energy  expenditure  was  significant,  while  in  35%  the 
savings  were  only  marginal. 

With  reference  to  capacity  building  and  employment,  the 
question  was  whether  additional  employment  had  been  created 
or  further  training  had  been  provided  since  the  completion  of  the 
projects.  The  results  point  out  that  on  the  one  hand  most  jobs  that 
had  been  planned  and  established  during  the  implementation 
phase  still  exist.  On  the  other  hand,  the  generation  of  additional 
employment  opportunities  after  the  initial  projects  were  com¬ 
pleted  tended  to  be  limited.  In  total,  eight  of  the  reviewed  projects 
reported  that  additional  people  have  been  trained  either  in 
managing,  constructing  or  repairing  the  renewable  energy  tech¬ 
nologies  or  in  handling  and  preforming  associated  tasks  like  food 
processing,  irrigation  or  business  activities.  Five  of  these  projects 
could  provide  numbers  on  how  many  people  were  trained  and 
potentially  employed,  meaning  that  although  the  employment 
potential  did  not  increase  significantly  after  project  completion,  at 
least  66  additional  people  received  training  and  the  chance  of 
employment  as  technicians  for  the  implemented  technologies  or 
in  the  associated  business  activities.  Depending  on  the  projects  the 
people  who  received  training  were  chosen  by  the  communities  or 
the  implementing  organizations  based  on  their  technical  abilities 
or  to  improve  existing  social  disadvantages  based  on  their  socio¬ 
economic  or  socio-cultural  position. 

Concerning  the  overall  quality  of  life,  most  small-scale  renew¬ 
able  energy  projects  claim  that  access  to  energy  will  improve  the 
quality  of  life  for  the  local  population.  But  quality  of  life  is  not  easily 
measured;  the  positive  impacts  already  noted  such  as  energy 
access,  reduction  of  energy  expenditure  and  employment  oppor¬ 
tunities  all  contribute  to  improving  the  living  conditions  in  local 
households.  Most  of  the  projects  continue  to  provide  energy 
services  that  were  previously  unavailable  and  have  helped  to 
reduce  household  expenditure  on  energy,  meaning  that  the 
evaluation  results  allow  us  to  draw  the  conclusion  that  the  living 
conditions  of  most  of  the  target  groups  improved  (at  least 
partially)  due  to  access  to  basic  energy  services,  clean  water  and 
lighting.  The  replacement  of  traditional  solid  fuels,  such  as  wood 
fuel  and  fossil  fuels  e.  g.  kerosene,  helped  to  reduce  indoor  air 
pollution,  improving  the  health  situation  particularly  for  women 
and  children.  At  the  same  time,  in  several  projects  the  local 
population's  awareness  of  health-related  issues  linked  to  indoor 
air  pollution  was  raised.  Apart  from  these  factors,  other  influences 
that  can  impact  significantly  on  living  standards,  such  as  a 
reduction  in  the  time  spent  on  gathering  fuel  wood  or  increased 
access  to  information,  communication  and  entertainment  services 
such  as  television,  radio,  mobile  phone  and  internet,  could  not  be 
explicitly  quantified  by  the  evaluation. 


4.1.2.  Impacts  of  projects  on  MDG  7:  ensuring  environmental 
sustainability 

Environmental  objectives  related  to  energy  use  in  developing 
countries  include  reducing  deforestation  by  reducing  the  use 
of  fire  wood,  preserving  biodiversity,  preventing  unsustainable 


land-use  changes  and  reducing  smoke  and  GHG  emissions  by 
replacing  wood  and  fossil  fuels  as  the  energy  source.  Accordingly, 
the  23  projects  supported  by  the  SEPS  scheme  should  have 
provided  local  and/  or  global  environmental  benefits  and,  at  the 
same  time,  should  not  have  produced  negative  environmental  side 
effects.  Therefore  the  technologies  introduced  in  the  initial  pro¬ 
jects  aimed  to  reduce  GHG  emissions  by  replacing  inefficient 
technologies  and  high  emission  technologies  that  use  fossil  fuels 
or  by  introducing  technologies  to  reduce  the  unsustainable  supply 
and  use  of  fuel  wood.  While  these  positive  environmental  effects 
have  been  quantified  (on  an  estimated  basis)  for  the  initial  project 
phase,  it  was  not  possible  to  provide  quantifiable  results  on  a 
sound  scientific  basis  from  the  review  data.  A  few  interview 
partners  supplied  figures  relating  to  the  post-project  period  for 
the  reduction  made  in  tones  of  GHG  emissions  and  wood  or  liters 
of  kerosene  and  diesel  -  but  these  were  solely  based  on  rough 
estimates.  Nevertheless,  as  the  majority  of  the  technological 
devices  introduced  in  the  projects  are  still  operating,  it  can  be 
expected  that  most  of  the  projects  have  succeeded  in  reducing  C02 
emissions  and  fossil  fuel  and  fire  wood  use  on  an  ongoing  basis. 
With  the  exception  of  one  project  which  led  to  unintended 
negative  effects  on  the  environment  as  farmers  chose  to  use  diesel 
to  fuel  their  new  irrigation  pumps,  instead  of  oil  from  wild 
Jatropha  seeds.  Apart  from  this  particular  project  (and  those 
projects  that  stopped  operating)  it  can  be  anticipated  that  most 
projects  have  had  some  positive  impact  on  ensuring  environmen¬ 
tal  sustainability  by  replacing  unsustainable  fuels  and  reducing 
GHG  emissions. 


4.1.3.  Impacts  of  projects  on  MDG  8:  promoting  global  partnership 
for  development 

Introducing  innovative  renewable  energy  technologies  to  local 
communities  is  not  only  about  providing  those  people  with  energy 
and  reducing  the  negative  impacts  of  unsustainable  fuel  use,  but  is 
also  about  spreading  knowledge  on  sustainable  energy  options, 
demonstrating  how  these  technologies  can  be  applied  in  the  local 
context  and  building  networks  to  promote  these  technologies. 
In  this  context,  small-scale  energy  projects  can  be  seen  as  a  first 
step  in  the  process  of  the  wider  dissemination  of  sustainable 
energy  technologies.  Significant  factors  include  establishing  net¬ 
works  and  partnerships,  promoting  interaction  at  political  level 
and  designing  strategies  for  dissemination  and  replication.  Thus 
following  section  looks  at  how  far  the  projects  addressed  these 
issues  and  what  contribution  they  made  to  building  networks  and 
partnerships  for  development. 

One  question  is  concerned  with  how  local  actions  are  affected 
by  national  and  international  systems,  strategies  and  policies  and 
vice  versa.  The  review  of  the  23  projects  emphasized  the  impor¬ 
tance  of  the  following  two  forms  of  interaction:  the  direct  negative 
effect  national  infrastructure  developments  can  have  on  local 
projects  and  the  positive  effects  small-scale  energy  projects  can 
have  on  local,  regional  and  national  energy  regulations.  Four  of  the 
projects  in  the  sample  were  negatively  affected  by  national  grid 
extension  programs;  the  result  in  most  of  these  cases  was  the 
abandonment  of  the  project,  as  households  newly  connected  to 
the  national  grid  dropped  their  commitment  to  the  decentralized 
energy  systems.  This  was  particularly  the  case  for  technologies 
that  are  not  cost  competitive  with  the  electricity  prices  and  the 
service  quality  of  24-h  power  supply  from  the  grid.  On  the  other 
hand,  it  was  discovered  that  when  successful  projects  are  com¬ 
municated  well  to  the  political  level  they  can  have  influence  on 
renewable  energy  regulations.  To  influence  regulations,  the  tech¬ 
nologies  introduced  must  be  cost-effective  in  relation  to  alter¬ 
native  energy  regimes  (e.g.  kerosene,  diesel  generator,  grid 
extension  to  remote  areas)  and  the  potential  for  the  country  must 
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be  proven.  Although  direct  causal  relationships  are  hard  to  prove, 
there  is  evidence  to  suggest  that  at  least  three  projects  stimulated 
to  some  extent  government  decisions  to  introduce  support 
schemes  -  such  as  feed-in-tariffs  or  cheap  loans  -  to  foster  the 
dissemination  of  the  respective  technologies.  One  example  was 
the  Latin  America  and  Caribbean  biogas  network  “Redbiolac”, 
which  facilitated  the  exchange  between  practitioners  and  decision 
makers.  The  exchange  in  the  network  and  at  the  workshops 
provided  decision  makers  with  evidence  on  the  feasibility  of  the 
biogas  technology  in  the  region.  This  gave  decision  makers  in 
Nicaragua  the  confidence  to  push  the  technology,  therewith 
supporting  the  introduction  of  a  national  biogas  program  in 
Nicaragua. 

In  order  for  an  individual  project  to  make  an  impact  outside  the 
local  context,  it  is  vital  that  the  project  succeeds  in  developing, 
extending  and  strengthening  partnerships  and  links  at  local, 
national  and  international  level.  Of  the  23  reviewed  projects, 
44%  stated  that  network  connections  between  parties  involved 
were  developed  or  strengthened.  Equally,  44%  reported  that  other 
organizations  within  the  region  had  become  involved  with  the 
technology.  The  importance  of  working  with  other  organizations 
and  exchanging  knowledge  and  information,  as  well  as  the 
presence  of  institutions  accompanying  the  introduction  of  new 
technologies,  was  stressed  by  several  interview  partners.  Two  of 
the  projects  declared  that  network  development  between  com¬ 
munities,  academic  institutions,  NGOs,  rural  and  national  govern¬ 
ment  organizations  etc.  was  one  of  the  most  important  impacts  of 
the  project.  On  the  other  hand,  the  lack  of  these  network  connec¬ 
tions  or  institutional  frameworks  led  to  difficulties  or  failure  for  at 
least  three  projects.  The  results  correspond  with  experience  from 
photovoltaic  implementations  [32,33],  which  found  that  interven¬ 
tions  with  a  clear  view  beyond  the  donor  commitment  period  had 
higher  sustainability  rates.  Implementing  organization  that  had 
strong  ties  to  the  region  and  were  not  only  involved  for  the 
implementation  period  often  proved  to  be  more  successful  in 
achieving  these  long-term  goals  [34].  A  study  on  the  scaling  up  of 
development  projects  in  Tadzhikistan  [35]  came  to  the  same 
conclusion,  describing  it  as  stick-with-it  mentality. 

With  regards  to  dissemination,  it  is  necessaiy  to  provide 
information  on  the  project  itself  and  the  applied  technology  in 
order  to  convince  local  stakeholders,  such  as  authorities,  funding 
institutions  or  project  developers  of  the  benefits.  This  can  help  to 
increase  not  only  the  acceptance,  which  is  vital  for  renewable 
energy  projects  [36],  but  also  the  replication  of  the  project.  In  the 
evaluation  sample  all  23  projects  had  strategies  in  place  to  inform 
stakeholders  and  spread  information  and  stated  at  the  outset  that 
the  replication  potential  was  high.  The  findings  of  the  evaluation 
show  that  these  dissemination  strategies  were  generally  suitable 
as  in  78%  of  the  cases  the  awareness  of,  and  interest  in,  renewable 
energy  technologies  was  increased.  But  an  information  system  is 
of  little  relevance  if  it  does  not  form  part  of  a  wider  action  system. 
The  review  data  revealed  that  the  potential  for  replication  was 
strongly  influenced  by  the  continuing  involvement  of  the  imple¬ 
menting  organization.  Overall  it  was  possible  to  replicate  either 
the  technology  or  the  management  model  in  40%  of  the  cases. 
Another  factor  that  was  mentioned  as  a  central  barrier  to  replica¬ 
tion  was  the  high  up-front  investment  costs  required  for  renew¬ 
able  energy  systems.  These  costs  meant  that  donor  funding  or 
other  forms  of  subsidies  were  needed  for  replication  in  all  but  one 
case.  This  shows  that  although  a  system  may  be  successful  and 
accepted  by  the  users,  replication  rarely  happens  on  its  own  but 
that  several  conditions  need  to  be  in  place  before  replication  is 
possible.  These  conditions  include  long-term  commitment  by  the 
implementing  organization,  strategies  with  an  explicit  focus  on 
replication  and  the  availability  of  additional  medium  to  long-term 
funding. 


4.2.  Sustainability 

The  first  question  this  evaluation  aimed  to  answer  with  regards 
to  sustainability  was  whether  or  not  the  energy  services  and 
structures  could  be  sustained  over  an  extended  period  of  time 
after  the  initial  project  was  completed.  As  described  in  Section  3.1 
the  indicator  applied  to  evaluate  sustainability  was  whether  the 
technology  was  still  functioning  and  in  use  at  the  time  of  the 
review.  The  findings  show  that  the  majority  of  the  23  projects 
(78%)  were  still  operating  and  in  use  by  the  beneficiaries.  Of  these 
projects,  48%  were  fully  functioning  and  30%  were  largely  opera¬ 
tional,  with  only  some  installations  or  structures  no  longer 
functioning.  A  further  13%  of  the  former  projects  were  only 
functioning  to  limited  extent,  while  9%  of  the  projects  failed 
completely  (Fig.  2).  Of  the  cases  that  failed  or  are  only  operational 
to  limited  extent,  technologies  using  biomass  as  renewable  fuel 
source  represented  the  largest  cluster  (80%).  All  projects  that 
aimed  to  meet  the  need  of  food  preparation  using  technologies 
that  need  less  energy  inputs  such  as  improved  cooking  stoves  and 
solar  cookers  are  still  fully  or  mostly  operational. 

The  second  question  the  evaluation  attempted  to  answer  with 
regards  to  the  sustainability  of  the  projects  was  what  the  critical 
factors  were  that  influenced  the  achievement  or  non-achievement 
of  sustainability.  In  answering  this  question  it  was  discovered  that 
the  extent  to  which  the  energy  services  were  sustained  depended 
on  a  number  of  factors  including  system  inherent  elements  such  as 
the  effective  functioning  of  the  technical  system,  the  financial 
viability  of  the  service  and  the  effective  management  of  the 
project,  as  well  as  external  factors  and  events  such  as  institutional 
and  policy  developments  or  environmental  conditions.  This 
demonstrates  that  there  are  a  wide  range  of  potential  factors  that 
influenced  the  success  or  failure  of  small-scale  energy  projects  in 
developing  countries.  Despite  this,  it  was  possible  to  identify 
several  common  factors  that  supported  the  sustainability  of 
projects  while  two  factors  were  recognized  as  being  instrumental 
in  leading  to  failure  of  the  reviewed  technology  interventions. 
Those  factors  that  positively  or  negatively  influenced  the  sustain¬ 
ability  of  the  23  projects  are  described  in  detail  in  the  following 
section  and  are  summarized  in  Table  4. 

In  over  70%  of  the  successful  cases,  significant  changes  were 
made  to  the  initially  proposed  project  design.  These  adaptations 
included  technical  changes,  modifications  in  finance  mechanisms 
or  adjustments  to  the  business  model  or  management  system.  The 
modifications  were  possible  because  the  implementing  organiza¬ 
tions  were  flexible  enough  to  adapt  to  practical  needs  which  arose 
during  the  implementation  and  afterwards.  In  detail,  adjustments 
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Table  4 

Summary  of  factors  that  influence  the  sustainability  of  small-scale  renewable  energy  projects. 


Positive  influences 


Negative  influences 


•  Local  availability  of  maintenance  and  repair  service 

•  Trust  and  reliability  between  implementing  organization  and  other  stakeholders 

•  Sense  of  ownership  among  beneficiaries  was  critical,  particularly 
in  community  projects 

•  User  satisfaction  with  technology 


•  Unsuitability  of  technology  in  geographic  context 

•  External  influences  such  as  political,  institutional  and  environmental  settings 

•  Low  motivation  of  potential  users/producers,  lacking  sense  of  ownership 

•  Problematic  logistics,  lack  of  capacity  to  manufacture  and  install 
technology  properly 


to  the  applied  technology,  major  repairs  or  selection  of  new 
components  were  reported  in  47%  of  the  projects  that  are  still 
mostly  or  fully  operational.  In  order  to  sustain  the  systems,  the 
availability  of  maintenance  and  repair  service  for  the  implemented 
technology  was  vital.  In  nearly  80%  of  the  successful  cases, 
maintenance  and  repair  services  were  at  least  partly  offered 
locally,  indicating  that  small  repairs  can  be  done  locally  without 
outside  support  or  time-consuming  ordering  of  replacement  parts. 
The  findings  further  indicate  that  availability  of  knowledge, 
expertise  and  skills  required  to  sustain  the  technical  systems  was 
central  to  the  technical  viability  of  the  projects.  This  observation  is 
in  line  with  the  findings  from  other  studies  [24,25,32],  which 
acknowledged  problems  resulting  from  lack  of  technical  capacity 
in  rural  areas  and  the  logistical  difficulties  of  servicing  equipment 
as  one  of  the  major  issues.  That  the  availability  of  local  main¬ 
tenance  service  is  not  only  an  important  factor  for  the  sustain¬ 
ability  of  small-scale  projects  but  also  one  of  the  concerns  that  the 
users  have  with  regards  to  renewable  technologies  was  shown  by 
Pode  [37]  who  studied  the  financing  of  LED  solar  home  systems  in 
developing  countries. 

The  integration  and  interconnection  of  the  implementing 
organization  within  the  region  was  also  identified  as  an  essential 
factor  to  support  sustainability.  While  the  integration  and  partici¬ 
pation  of  the  beneficiaries  is  already  an  important  selection 
criterion  for  SEPS  support,  the  evaluation  provides  indications 
that  links  between  the  implementing  organization  itself  and  local, 
regional  and  national  institutions,  governmental  organizations  and 
other  local  stakeholders  are  equally  important.  This  factor  is 
described  by  one  interview  partner  as  being  in  it  for  the  long  run; 
he  further  pointed  out  the  importance  of  trust  and  reliability  on 
both  sides:  the  implementing  organization  as  well  as  the  local 
partners.  Consequently,  poor  communication  and  cooperation 
between  the  project  team,  the  users  and  the  institutions  was 
identified  as  a  major  factor  that  put  the  sustainability  of  a  project 
at  risk.  It  was  also  demonstrated  that  this  ongoing  involvement 
was  not  only  important  for  the  projects'  sustainability,  but  also  a 
vital  factor  for  the  replication  potential  of  the  renewable  energy 
systems.  In  all  replicated  cases  the  organizations  continued  to 
actively  support  the  technology  and  were  connected  to,  or  present 
in,  the  region. 

In  addition  to  the  characteristics  already  mentioned,  it  was 
recognized  that  two  other  factors  affected  the  sustainability  of  the 
projects:  the  sense  of  ownership,  referring  to  the  commitment  of  the 
beneficiaries  to  the  project  and/or  the  technology,  and  how  satisfied 
the  users  were  with  the  technology  in  terms  of  energy  supply, 
reliability  and  the  daily  operation  of  the  systems.  These  results  are  in 
line  with  the  existing  literature  on  sustainable  energy  interventions, 
identifying  ownership  as  one  of  the  factors  that  contribute  to  the 
sustainability  of  rural  electrification  projects  [38],  The  23  projects 
provided  evidence  that  ownership  was  less  of  a  problem  if  indivi¬ 
dual  households  were  the  owners  and  took  responsibility  for  the 
technology,  as  in  the  case  of  improved  cooking  stove  or  solar 
cookers.  In  the  case  of  efficient  and  improved  lighting  projects  in 
institutions  like  universities  or  hospitals  ownership  was  also  not  the 
most  pressing  issue  because  the  actual  beneficiaries  (e.g.  students 


and  staff)  did  not  have  to  take  care  of  the  systems.  But  for 
community  projects  ownership  proved  to  be  a  critical  factor  for 
the  sustainability  of  the  development  intervention.  In  certain 
projects  the  absence  of  ownership  was  a  major  factor  contributing 
to  an  unsustainable  outcome.  These  factors  proved  to  be  especially 
critical  with  in  bioenergy  projects.  An  example  is  the  case  of  a  biogas 
project  where  due  to  the  lacking  sense  of  ownership  nobody  felt 
responsible  for  the  supply  of  the  substrates,  resulting  in  underuse  of 
the  technology  and  conflict  between  the  community  members. 

In  terms  of  user  satisfaction  with  the  implemented  technology, 
in  82%  of  the  projects  the  users  were  at  least  partly  satisfied.  Of 
these,  65%  were  content  and  the  remaining  17%  were  partly 
content.  In  light  of  this  high  level  of  user  satisfaction,  the  findings 
support  the  assumption  that  in  small  small-scale  energy  projects  it 
is  rarely  the  technology  on  its  own  that  defines  the  sustainability 
of  a  project. 

With  regards  to  funding  opportunities,  the  question  addressed  was 
to  which  extent  the  projects  could  encourage  financial  institutions, 
government  or  community  organizations  to  provide  adequate  finance 
options  for  the  supported  energy  technologies  [20],  As  already 
described,  such  finance  options  are  vital  for  replication  and  dissemina¬ 
tion  because  most  potential  users  cannot  afford  the  technology  with¬ 
out  some  kind  of  support  The  evaluation  findings  show  that  in  35%  of 
the  reviewed  cases  some  kind  of  financing  is  now  available.  These 
finance  options  vary  widely,  from  feed-in  tariffs  to  loans  from 
governmental  or  community  organizations.  In  some  cases  financing 
is  in  theory  available  but  in  reality  the  poor  find  it  difficult  to  actually 
obtain  these  loans  or  the  people  are  not  willing  to  make  high  upfront 
investment  in  a  technology  that  is  new  to  them.  An  alternative  finance 
option  was  further  donor  funding.  With  the  successful  demonstration 
of  the  technology  in  the  project  supported  by  WISIONS,  some 
organizations  were  then  able  to  secure  larger  grants  from  other 
sources.  Due  to  the  high  upfront  investment  required  for  some 
renewable  energy  systems  this  is  often  the  only  realistic  way  to 
promote  these  technologies  in  an  early  stage. 

One  organization  mentioned  a  different  aspect  with  regards  to 
financial  sustainability.  Their  experience  was  that  most  people  in 
their  project  areas  were  not  used  to  credit  periods  longer  than  one 
year,  meaning  that  they  hesitated  to  make  the  type  of  long-term 
investment  required  for  some  energy  systems.  Therefore,  for 
certain  technologies,  finance  models  where  people  do  not  have 
to  make  large  upfront  investments  but  pay  small  amounts  for 
renting  the  equipment  may  be  an  option.  In  this  way  users  do  not 
take  the  risk  of  paying  the  whole  system  without  knowing  if  it  will 
deliver  the  promised  services. 

When  examining  the  factors  that  lead  to  unsustainable  devel¬ 
opments  the  evaluation  results  show  that  four  of  the  five  projects 
that  ceased  operating  or  are  only  functioning  to  a  limited  extent, 
were  using  biomass  as  energy  source,  while  one  project  implemen¬ 
ted  photovoltaic  and  micro-hydro  power  technology  for  electrifica¬ 
tion.  Analyzing  the  details  that  of  what  caused  these  projects  to 
be  unsustainable  it  becomes  obvious  that  one  factor  all  biomass 
projects  struggled  with  was  to  supply  the  needed  substrates. 
Although  the  causes  for  the  insufficient  supply  of  biomass  substrate 
differed,  the  conclusion  can  be  drawn  that  the  technology  was  not 
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appropriate  in  the  social  and  geographic  context.  These  undesirable 
experiences  with  bioenergy  projects  correspond  to  findings  from 
Han  et  al.  [20]  on  small-scale  bioenergy  in  China. 

The  second  factor  identified  as  critical  for  the  unsustainable 
developments  was  the  influence  of  policy  developments,  environ¬ 
mental  surroundings,  technology  advances  and  institutional  set¬ 
tings.  Because  projects  are  not  implemented  in  perfect  isolation, 
these  external  elements  are  highly  influential  for  success  or  failure. 
As  already  described  in  Section  4.1.3,  the  evaluation  results 
provide  evidence  of  the  direct  effects  that  national  infrastructure 
developments  can  have  on  local  projects.  In  addition  to  these  two 
factors  other  barriers  that  were  mentioned  to  influence  sustain¬ 
ability  include  the  low  level  of  motivation  of  potential  producers/ 
users,  problematic  logistics  in  remote  areas  and  a  lack  of  local 
capacity  to  manufacture  and  install  the  technology  properly. 


5.  Research  outcomes  and  recommendations 

Our  research  indicates  that  although  the  23  projects  implemented 
different  renewable  energy  technologies  in  different  socio-econom- 
ical,  environmental  and  geographical  contexts  with  different  man¬ 
agement  models  and  finance  schemes,  parallels  can  be  drawn  with 
regards  to  impacts  and  factors  that  influence  mid-term  sustainability. 
When  the  impacts  of  these  small-scale  renewable  energy  projects  are 
measured  against  the  MDGs,  most  projects  show  similar  develop¬ 
ments  in  terms  of  energy  access,  energy  expenditure,  productive  uses 
and  awareness-raising.  With  regards  to  sustainability,  the  results 
support  the  assumption  that  sustainability  does  not  only  depend  on 
the  reliability  of  the  technology  itself  but  also  on  a  sense  of  owner¬ 
ship,  user  satisfaction,  availability  of  adequate  knowledge  and  skills 
as  well  as  network  connections  and  the  commitment  of  the 
implementing  organization.  Therefore,  the  remaining  question  to 
be  answered  is  what  lessons  can  be  learned  to  improve  future 
development  interventions  in  terms  of  impacts,  sustainability  and 
the  opportunities  for  dissemination  and  replication. 

There  is  no  doubt  that  projects  must  be  context  sensitive  and 
adapted  to  local  needs,  yet  the  following  universally  valid  recom¬ 
mendation  can  be  made  to  improve  the  selection  process,  project 
design,  implementation  phase  and  follow-up  of  future  projects 
based  on  the  lessons  learned  from  the  comparison  of  the  23 
projects  presented  in  this  paper: 

•  First  of  all  it  is  important  to  ensure  that  the  implementing 
organization  is  committed  to  the  region  and/or  the  technology 
beyond  the  length  of  the  project.  Ideally  the  organization 
should  be  embedded  in  the  local  context. 

•  Even  more  emphasis  should  be  put  on  the  motivation  and 
involvement  of  the  potential  users.  It  has  to  be  assured  that 
they  develop  a  sense  of  ownership,  even  when  the  equipment 
does  not  belong  to  an  individual  but  to  a  community. 

•  Especially  for  technologies  that  use  biomass  as  energy  source, 
detailed  analysis  of  the  availability  and  supply  of  substrate 
should  be  conducted.  Estimates  have  proved  to  be  insufficient 
for  ensuring  a  continuous  substrate  supply  for  an  actual  project. 

•  The  expectations  regarding  productive  uses  and  business 
development  initiated  by  renewable  energy  projects  at  local 
level  should  not  be  too  high,  as  the  potential  for  development 
in  this  area  has  been  overestimated.  The  project  design  must 
explicitly  incorporate  activities  that  go  beyond  energy  access  in 
order  for  these  to  be  an  outcome  of  the  project. 

•  Instead  of  addressing  only  energy  related  needs,  other  needs 
such  as  local  environmental  protection,  sustainable  agriculture 
and/or  capacity  building  should  be  addressed  jointly. 

•  To  improve  the  need  and  impact  assessment,  a  baseline  study 
on  the  status  quo  of  regional  conditions  should  be  developed 


for  each  project  prior  to  implementation.  This  would  allow  for 
better  quantification  of  project  outcomes. 

•  As  well  as  project  monitoring  during  the  implementation 
phase,  post-evaluations  should  become  an  integrated  part  of 
project  concepts.  At  least  one  ex-post  evaluation  should 
become  mandatory. 

•  Regional  networks,  partnerships  and  South-South  cooperation 
should  be  intensified  because  projects  run  by  organizations  with 
established  extensive  network  connections  and  good  links  to  the 
region  and  beneficiaries  proved  to  have  higher  success  rates. 

In  conclusion,  the  majority  of  the  23  projects  had  positive 
effects  on  sustainable  development  that  were  measurable  against 
the  MDGs,  although  there  is  no  single  solution  for  ensuring  that 
projects  produce  positive  impacts.  However,  this  evaluation  has 
demonstrated  that  energy  projects,  even  if  only  small-scale  cannot 
be  one-time  development  assistance  but  require  continuously 
involvement  and  further  support  (as  opposed  to  one-off  support). 

6.  Discussion  and  conclusion 

We  conclude  by  discussing  the  limitations  and  implications  of 
the  presented  study  results.  One  of  the  main  constrains  is  the 
limited  scope  of  the  study.  On  the  on  hand  the  projects  were  all 
supported  under  the  same  scheme,  meaning  that  they  were 
already  selected  based  on  criteria  that  are  sensitive  to  the 
problems  occurring  in  small-scale  energy  projects.  Therefore  the 
high  sustainability  rate  might  not  be  representative  for  these  types 
of  development  interventions.  On  the  other  hand  the  number  of 
projects  was  limited.  Although  many  impact  evaluations  and 
sustainability  assessments  focus  on  an  even  smaller  number  of 
projects,  due  to  the  cross-sectional  nature  of  the  study  a  larger 
number  of  projects  analyzed  under  the  same  framework  could 
improve  the  robustness  and  validity  of  the  result.  A  third  potential 
constraint  to  the  study  stems  from  the  fact  that  the  environmental 
dimension  of  sustainability  could  not  be  fully  assessed.  Environ¬ 
mental  indicators  such  as  the  reduction  of  C02  emissions,  avoided 
conventional  fuel  use,  reduction  of  deforestation  or  preservation  of 
biodiversity  are  more  difficult  to  measure  and  to  quantify  than  e.  g. 
numbers  of  installations  or  people  trained.  As  the  projects  sup¬ 
ported  by  W1SIONS  are  small-scale  with  limited  budgets  it  is  not 
possible  to  provide  quality  data  on  these  types  of  indicators,  it  is 
therefore  difficult  to  determine  the  full  environmental  impact. 

Despite  these  limitations,  the  results  give  a  first  indication  on  how 
the  sustainability  of  energy  interventions  at  the  local  level  can  be 
improved.  However,  further  research  is  required  especially  if  replica¬ 
tion  and  dissemination  are  to  be  achieved.  The  limited  availability  of 
data  is  still  hindering  the  research  progress.  Therefore,  sustainable 
energy  projects  in  general,  including  those  on  a  small-scale,  addres¬ 
sing  households  or  communities,  should  be  systematically  evaluated 
in  regular  intervals.  Hereby  sound  empirical  evidence  on  differen¬ 
tiated  welfare  and  empowerment  gains  and  the  conditions  under 
which  these  interventions  have  the  greatest  impact  can  be  gathered. 
Moreover,  the  results  imply  that  research  needs  to  be  carried  out  not 
only  to  identify  the  technical  potentials  in  rural  areas,  but  to  show 
how  local  stakeholders  can  be  better  motivated  to  use  the  energy  for 
economic  activities  and  how  the  replication  potential  of  projects  can 
be  enhanced.  In  connection  with  the  replication  and  dissemination 
potential  another  area  calling  for  closer  assessments  is  the  financing 
of  local  energy  access.  It  is  necessary  to  further  investigate  how  to 
provide  adequate  finance  mechanisms  to  the  poor  of  the  poor  and  to 
communities  as  a  whole  to  prevent  problems  associated  with  the 
limited  length  and  amount  of  funding.  Finally  the  research  should 
not  only  focus  on  impacts  in  isolation,  but  on  the  transition  processes 
occurring  within  the  socio-technical  systems  that  lead  to  these 
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impacts.  In  the  end  impacts  and  sustainability  of  energy  interven¬ 
tions  do  not  stand  alone,  but  are  a  result  of  the  changes  in  the  socio¬ 
economic  configurations. 
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